A Comparison of Event Models for Naive Bayes Text Classi cation

Recent approaches to text classi cation have used two di erent rst order probabilistic models for classi ca tion both of which make the naive Bayes assumption Some use a multi variate Bernoulli model that is a Bayesian Network with no dependencies between words and binary word features e g Larkey and Croft Koller and Sahami Others use a multinomial model that is a uni gram language model with integer word counts e g Lewis and Gale Mitchell This paper aims to clarify the confusion by describing the di erences and details of these two models and by empirically comparing their classi cation performance on ve text corpora We nd that the multi variate Bernoulli performs well with small vocabulary sizes but that the multinomial performs usually performs even better at larger vocabulary sizes providing on average a reduction in error over the multi variate Bernoulli model at any vocabulary size